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On the Perfonnance of Turbo Signal Recovery 
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Abstract —This letter is on the performance of the turho signal 
recovery (TSR) algorithm for partial discrete Fourier transform 
(DFT) matrices based compressed sensing. Based on state evo¬ 
lution analysis, we prove that TSR with a partial DFT sensing 
matrix outperforms the well-known approximate message passing 
(AMP) algorithm with an independent identically distrihnted 
(IID) sensing matrix. 

Index Terms —T\irho compressed sensing, signal recovery, 
AMP, partial DFT, state evolution. 


I. Introduction 


The approximate message passing (AMP) algorithm ||]J-|j9| 
is an efficient signal recovery method for compressed sens¬ 
ing. Its convergence is asymptotically guaranteed for sensing 
matrices with independent identically distributed (IID) entries 
using the state evolution technique 0- The fixed points 
of the state evolution for AMfQ include the optimal minimum 
mean squared-error estimation (MMSE) solution |Ti)-|Tg. 
This indicates that AMP is asymptotically optimal when the 
state evolution equation has a unique solution. 

AMP can also be applied to problems involving non-IID 
sensing mattices Oi), fig. However, the state evolution 
technique is not directly applicable in this case. 

Alternative techniques have been developed for non-IID 
sensing matrices |Tg-|[T8). It has been observed that these 
techniques with partial discrete Fourier transform (DFT) matri¬ 
ces |Tg-@ can outperform AMP with IID sensing matrices 
under proper normalization conditions. The comparisons in 
m were based on simulations and no analytical results 
have been reported so far. 

This letter is on the performance analysis of turbo signal 
recovery (TSR) with a partial DFT sensing matrix ID- We 
prove based on state evolution that TSR with a partial DFT 
matrix (TSR-DFT) outperforms AMP with an IID Gaussian 
matrix (AMP-IID). Since the state evolution technique for 
AMP does not apply to problems involving a partial DFT 
matrix (AMP-DFT), we compare TSR-DFT and AMP-DFT 
through simulations. Our numerical results suggest that TSR- 
DFT converges faster than AMP-DFT. 
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^Throughout this letter, AMP refers to AMP-MMSE [lo) . 


II. Problem Description 
Consider the following linear system: 

y — -^partial^ 71 (1) 

where x C is a sparse signal, n ^ CN'{0^a^I) the 

additive white Gaussian noise (AWGN) and .Fpartiai C 
(M < A^) a partial DFT matrix consisting of M randomly 
selected rows of the normalized DFT matrix F. The (m, n)th 
entry of F is given by ^ exp ( _ j . 

We assume that the entries of x are IID. The jth entry Xj 
follows the Bernoulli-Gaussian distribution 0 

{ 0 probability = 1 — A, 

( 2 ) 

CM{0,\ probability = A. 

By this definition, = 1. Here A determines the sparsity 

of the system. The partial DFT matrix can be rewritten as 

F’partial = SF, (3) 

where S consists of M randomly selected rows of the identity 
matrix. We define the following auxiliary vector: 


2 = Fx. (4) 

Combining 0 and Q, we have 

y — Sz + n. (5) 


Our objective is to recover x based on y under the assumption 
that X is sparse with A < 1. 


Module B 


Module A 



Fig. 1. Functional diagram of a standard turbo processor. 


III. Turbo Signal Recovery 
A. Standard Turbo Processor 

Fig. [T] shows a standard turbo-type signal processor | |23) for 
the problem under consideration. The related operations can 
be grouped into two modules labeled as A and B. Module A 
is a linear minimum mean-squared error (LMMSE) estimator 
1241 of tc based on Q without the sparsity information, while 
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Module B Module A 



Fig. 2. Functional diagram of the turbo signal recovery (TSR) algorithm d) .“ext” represents extrinsic message computation. 


module B estimates 2 ; based on the sparsity information in 
The two modules work iteratively. 

Since LMMSE estimation is standard, we will f 


module B. The input of module 
Fig. [T]), is modeled as ID 


B, denoted by x 


prt 


pri 


= X 


w, 


4) Compute the extrinsic mean/variance of x 


i on 

pri 

^r- 

ext 

(see 


Va 

(6) 

pri 

Xb 


ext 

Xa 




post 


pri 


■I— -71®^* 


post 

post 


pri 


(lOa) 

(10b) 


where w is IID Gaussian and independent of x. For each 
j, the sparsity combiner produces the a posteriori mean 
F,xj{xj\x^^} based on the AWGN assumption in (|^ and the 
sparsity constraint in (|^. Let denote indices excluding j. 
The extrinsic mean is dehned as F,a;^{xj\x^^^j}. Since is 
assumed to be an AWGN observation of x, the extrinsic mean 
will not improve during the iterative process based on Fig. 
The problem here is that the sparsity constraint is symboTby- 
symbol and so x^^^- does not provide any information about 
Xj. For details, see | [l8) . 

B. Turbo Signal Recovery 

The TSR algorithm proposed in is listed in Algorithm 
1 and graphically illustrated in FigTl^ The TSR algorithm 
computes the extrinsic message of 2 ; (instead of x) for module 
B. This avoids the above mentioned problem for the standard 
turbo processor. Refer to m for more details. 


5) Update 

zp;^^FxPj^\ ( 11 ) 

6) Compute the a posteriori mean/variance of each Xj 


XgJ ^ Ea;- 


7) Compute the a posteriori mean/variance of 2 ; 
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8) Compute the extrinsic mean/variance of 2 : 
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(14a) 


Algorithm 1: Turbo Signal Recovery (TSR) 

Initialization: 0, and ^ 1. 


for iteration = 1 : Tmax 

1) Update 
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j)ri 


^ F z 


H pri 


2) Compute the a posteriori mean/variance of z 
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where denotes the (j, j)th entry of S^S. 

3) Compute the a posteriori mean/variance of x 
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end 


IV. State Evolution Analysis 


In the following, we analyze the state evolution of TSR-DFT 


1181, based on which we prove that TSR-DFT outperforms 
AMP-IID. 


A. MMSE Properties for an AWGN System 

Assume that x has zero mean and unit variance. Consider 
the following observation of x corrupted by AWGN, 

r = X + w, (15) 

where w ^ CJV{Q,ri~^) is independent of x and p is the 
signal-to-noise ratio (SNR). Following p5) , dehne 

vaYx{x\r} = Yj^^\x - Ea;{a;|r}|^|r|, 


(16) 
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and 


minse(77) = Ej.{vara;{a;|r}}. 


(17) 


The following properties of mmse( ) are due to |25 Proposi¬ 
tions 4 and 9]: 


1 


(18a) 


Propertyl : mmse(ry) < —, 

- V 

Property2 : =-E,{(var,{a:|r})"}. (18b) 

The above two properties are useful to our later discussions. 


B. State Evolution of TSR-DFT 

We use the a priori variances and to measure the 
reliabilities of in 0 and in ( |10b| i, respectively. Our 
basic assumption is that x^^ in ( |10b| l is an AWGN observation 
of x: 

x^’' = X + w, (19) 

where w ^ CA/'(0, is independent of x. 

Define 

■u = and p = —. (20) 

'^B 

It is shown in p8j that the state evolution equations of TSR 
are given by 

I tv 2 ’ 

M M " 

= f-, (21b) 

where the superscripts represent the iteration indices, with 
initialization = 1. 


C. Convergence of State Evolution for TSR-DFT 

Proposition 1: ff) and tpf) in ( [2l] i are non-increasing 
functions. 

Proof: It is straightforward to see that ff) in ( |2Ia| ) is 
a non-increasing function of u*. We now rewrite pib| l as 
where 




o‘+i 


mmse 


( 22 ) 


So, 


d mmse 


d77‘+^ [mmse(p‘+i)]^ d?7*+i 

From Property 2 in ( |18b| ), we have 

d/ _ Er{(var)^} - (E^ {var}) 

(Eqt+i 




-1. (23) 


(Er {var}) 


> 0 


(24) 


where var = var^; {a;|r = a;-f raj and w ^ CN{0,rf'^^). 
From (0, /(•) is a non-decreasing function and so fif) = 
[/(•)]“^ is a non-increasing function. ■ 


Based on the monotonicity of the state transfer functions 
(pf) and V'(’)’ d can be proved that ju*} and {r]^~^^} are 
monotone, i.e.. 


v° >v^ > ■■■ >v°° andr]^ <r]^ < ■ ■■ <T]°^. (25) 


In the first iteration, f = 0 in ( |21a| l so 

'N-M N 


= 1 and 


M 


M 


-1 


> 0 . 


Applying Property 1 in ( |18a| l to pib| l yields 

1 


^mmse {ri°°) 
Combining ( |27l l and ( |21a| ), we have 

M 


-1 

' >0. 


-1 


V s 

Finally, from ( |25| l and (|27ll-(|28]l, we get 
1 = > . . . > > 0 , 


0 < r]^ < ■ ■ ■ <r]°° < 


M 1 

(V ' ct2- 


(26) 

(27) 

(28) 

(29a) 

(29b) 


From ( |29| l, the state sequences {u*} and are mono¬ 

tonic and bounded, and so they converge. Combining ( |21a| i and 
Plb| i, the stationary value r]°° is the solution of the following 
equation fTS): 


ri = 


mmse -I- cr^ — ^ (mmse -f cr^)^ — dcr^ • mmse • ^ 

5 

(30) 


(T^ • mmse 


where mmse is an abbreviation for mmse(? 7 °°). Note that ( |30| ) 
is consistent with the optimal MMSE performance obtained by 
the replica method. See p3l Eqns. (17) and (37)]. 


D. Comparison of TSR-DFT and AMP-IID 

Refer to the discussions in the Introduction. We now com¬ 
pare TSR-DET and AMP-IID based on their state evolution 
equations. 

The state evolution of AMP-IID is given by |[T0] Eqn. (41)], 
Eqns. (18) and (20)f] 

1 


^AMP-IID “ jy t 

M ’ ^AMP-IID 

^AMP-IID = mmse (77^ 


M " 


AMP-IID/ > 


with initiation riAMP-iiD = 1- 


Eor TSR-DET, we rewrite ( |2T| l as 

1 


N 


N-M 


^tIr-dft ~ jy 

M 


^TSR-DFT 


U 


TSR-DFT ■ 
1 


M " 


mmse (t/x^r.dpi-) 


’Itsr-dft 


(31a) 

(31b) 

(32a) 

-1 

I ■ 

(32b) 


The following helps to see the equivalence of 0 and ( [ 3 ^ : 

ffZNff ' ^TSR-DFT = 7 t^R-DFT = V ( 33 ) 

A factor of N/{N — M) is used ( |3^ to match ( |32a| l with 
(|31ali, which facilitates the proof of the proposition below. 


^Note that the variances of the entries of the IID Gaussian matrix are 
1/N, instead of 1/M as assumed in This is for the convenience of 

comparison with TSR-DFT. 
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Proposition 2: f^SR-DPT — ^amp-iid> for t > 0. 

Proof: We prove by induction on t. The initial conditions 

are t^AMP-iio = 1 ( [^ , 

n N-M n N-M „ 

%SR-DFT — — jy < %MP-IID- (34) 

Now suppose 

^TSR-DFT ^ 1'aMP-IID- (35) 


TSR-DFT outperforms AMP-IID in terms of estimation MSE 
in each iteration. 

Corollary 1: mmse (?7x^r-dft) — rnmse (pamp-iid)- 
Proof: By comparing ( |31a| l and ( |32a| l, together with 
Proposition 2, it is straightforward to see that > 

iiD- Corollary 1 follows since mmse(-) is a monotonically 
decreasing function. ■ 


It suffices to prove that 




<!^ £. 

TSR-DFT — AMP-IID- 


Combining ( |32a[ ) and ( |32b[ ), we have 

1 


N 


i+1 


N-M ^TSR-DFT 


= mmse 


+ (^-- 

N 


N t - 2 

^«TSR-DFT+ 


TSR-DFT 
-V 




(36) 


(37a) 


(37b) 


From (|25]l and ([33|l we have 


t+i 


^TSR-DFT — "^^TSR-DFT 

Replacing u* by in ( |37a| i, and using 
following inequality 

1 


N 


i+1 


N-M ^TSR-DFT 


(38) 

we obtain the 

-- (39a) 


^ \M 


V. 


i+1 

TSR-DFT 




< 


N 


i+1 


N-M ‘^TSR-DFT 


N 


M 


After some manipulations of (|39a|l, we get 


(K.M ,k 

1 M ‘^TSR-DFT ^ M 



(39b) 

i-n 

(39c) 


^TSR-DFT ^ 


^ ^TSR-DFT 

(40a) 

N 

M 

• 4sR-DFr + ""m"" ■ 

( f N 


N f\~^\ 


< mmse vr • 

\\M 

^TSR-DFT 3" ^ ^ J 1 • 

(40b) 


From ( |40l i and noting the fact that VxIr.qpx > 0 (from 
and ([33|), we have 

-i\ 


4sr-dft < mmse ( ( ^ ' Wtsr-dft + ^ ' 0"^ I • (41) 


'^TSR-DFT 


Now consider AMP-IID. Combining (|31a|i and (|31b[), we have 


^AMP-IID ~ minse 


N t ^2 

M ' ^ M ■ 


. (42) 


Note that mmse(-) is a monotonically decreasing function. 
Comparing ED and ( |42] i and based on the assumption that 
lixsR-DFT < ^Imp-iid^ we readily obtain < u^^iViid^ 

which proves ( |3^ . ■ 

The MSE performances of TSR-DFT and AMP-IID 
at iteration t are characterized by mmse (i?t^r.oft) 
mmse (pamp-iid) ’ respectively. Corollary 1 below shows that 


V. Numerical Examples 

Fig. shows the numerical results for AMP-IID, AMP-DFT 
and TSR-DFT. First, we see that the simulation and evolution 
results for TSR-DFT and AMP-IID agree very well. Note that 
only simulation results are provided for AMP-DFT since no 
efficient analysis technique is available. 

From Fig. we see that TSR-DFT outperforms AMP-IID in 
terms of both convergence speed and convergent MSE, which 
verifies Corollary 1. Also, the simulation results show that 
TSR-DFT converges faster than AMP-DFT. From Fig. it 
seems that the differences in the convergent MSEs are minor 
for TSR-DFT and AMP-DFT. However, if we decrease M, a 
more signihcant gain of TSR-DFT over AMP-DFT could be 
observed, see |18 Fig. 3]. 

In simulations, we hnd that the performance advantage of 
TSR over AMP shrinks as A decreases. We will not show the 
results here due to space limitation. 



Lig. 3. MSE performances of TSR and AMP. A = 0.4. N = 8192. M = 
5734 (r; 0.7N). The simulated MSEs are obtained by averaging over 500 
realizations. 


VI. Conclusions 

In this letter, we proved based on state evolution that TSR- 
DFT outperformed AMP-IID. In addition, our simulation re¬ 
sults suggest that TSR-DFT converges faster than AMP-DFT. 
Possible future work includes extending the TSR algorithm to 
the IID setting and compare it with AMP-IID. 
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